International Journal of COPD 



Q Open 



Access Full Text Article 



Dovepress 

open access to scientific and medical research 

ORIGINAL RESEARCH 



Development and validation of a model to predict 
the risk of exacerbations in chronic obstructive 
pulmonary disease 



Loes CM Bertens 1 
Johannes B Reitsma 1 
Karel GM Moons' 
Yvonne van Mourik 1 
JanWillemJ Lammers 2 
Berna DL Broekhuizen 1 
Arno W Hoes' 
Frans H Rutten' 

'Julius Center for Health Sciences 
and Primary Care, 2 Department of 
Pulmonology, Heart-Lung Center, 
University Medical Center Utrecht, 
Utrecht, the Netherlands 



Correspondence: Loes CM Bertens 

Julius Center for Health Sciences and 

Primary Care, University Medical Center 

Utrecht, PO Box 85060, Stratenum 

6. 1 3 1, 3508 AB Utrecht, the Netherlands 

Tel +3 1 88 756 8375 

Fax +3 1 88 756 9028 

Email l.c.m.bertens-2@umcutrecht.nl 



This article was published in the following Dove Press journal: 
International Journal of COPD 
9 October 2013 

Number of times this article has been viewed 



Purpose: Prediction models for exacerbations in patients with chronic obstructive pulmonary 
disease (COPD) are scarce. Our aim was to develop and validate a new model to predict 
exacerbations in patients with COPD. 

Patients and methods: The derivation cohort consisted of patients aged 65 years or over, 
with a COPD diagnosis, who were followed up over 24 months. The external validation cohort 
consisted of another cohort of COPD patients, aged 50 years or over. Exacerbations of COPD 
were defined as symptomatic deterioration requiring pulsed oral steroid use or hospitalization. 
Logistic regression analysis including backward selection and shrinkage were used to develop the 
final model and to adjust for overfitting. The adjusted regression coefficients were applied in the 
validation cohort to assess calibration of the predictions and calculate changes in discrimination 
applying C-statistics. 

Results: The derivation and validation cohort consisted of 240 and 793 patients with COPD, 
of whom 29% and 28%, respectively, experienced an exacerbation during follow-up. The final 
model included four easily assessable variables: exacerbations in the previous year, pack years 
of smoking, level of obstruction, and history of vascular disease, with a C-statistic of 0.75 (95% 
confidence interval [CI]: 0.69-0.82). Predictions were well calibrated in the validation cohort, 
with a small loss in discrimination potential (C-statistic 0.66 [95% CI 0.61-0.71]). 
Conclusion: Our newly developed prediction model can help clinicians to predict the risk of 
future exacerbations in individual patients with COPD, including those with mild disease. 
Keywords: exacerbation of COPD, risk prediction, external validation, vascular disease 

Introduction 

Exacerbations of chronic obstructive pulmonary disease (COPD) have a large impact 
on morbidity and quality of life and place a huge financial burden on the health care 
system. 13 Patients with moderate-to-severe COPD seek medical attention on average 
two to three times a year for exacerbations, of which one requires hospitalization. 4 This 
fuels the interest in strategies to reduce the number of exacerbations. Adequate predic- 
tion models could guide clinicians in subsequent intensified monitoring and treatment 
of those at increased risk and thus, reduce the number of exacerbations. The majority 
of published prediction models developed for patients with COPD, however, aim to pre- 
dict mortality, 5 ' 7 and models specifically developed to predict exacerbations are scarce. 
Moreover, most of these models are based on severe exacerbations, those requiring 
hospitalization. 8 " 10 Also, studies that developed prediction models for future exacerbations 
typically included severely diseased COPD patients recruited from tertiary care centers, 
and are therefore not representative of the COPD patient population at large. 
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Severity indices like the BODE (body mass index, 
obstruction, dyspnea and exercise) Index 5 and the DOSE 
(dyspnea, obstruction, smoking and exacerbations) Index 11 
have also been tested on their ability to predict future exac- 
erbations and showed acceptable performance measures (ie, 
C-statistics in the range of 0.6-0.7 in their original cohorts). 
As these scores were not initially developed to predict exac- 
erbations, some potentially important predictors for exacer- 
bations were not considered. Thus, the need remains for an 
easily applicable prediction model that can be applied to all 
patients with COPD, including those with mild disease and 
predicting all exacerbations, including those not requiring 
hospitalization. Also, the model should be practicable in 
everyday care, including the primary care setting, in which 
many patients with (mild) COPD are managed. We set out 
to develop and externally validate such a model. 

Materials and methods 

Study design and participants 

For the development of the prediction model, we used a 
cohort of COPD patients, originally set up to screen for 
heart failure in patients with COPD. 12 In brief, patients 
from primary care enlisted with a general practitioner's 
(GP's) diagnosis of COPD and aged 65 years or over were 
selected from 5 1 general practices in the Netherlands between 
April 2001 and June 2003. At the baseline assessment, the 
GP's diagnosis of COPD was reevaluated by newly performed 
standardized pulmonary function tests (spirometry, body 
box measurements, and diffusion testing), and presence 
or absence of COPD was determined by an expert panel 
including a pulmonologist, applying the Global initiative 
for chronic Obstructive Lung Disease (GOLD) criteria. 13 
COPD was established in 243 patients. Follow-up data for 
all 243 patients was collected over a period of 24 months 
from study entry measurements by scrutinizing the treating 
GP's electronic medical files, including letters from medical 
specialists. 14 The institutional review board of the University 
Medical Center Utrecht, Utrecht, the Netherlands, approved 
the study protocol, and all participants gave written informed 
consent. 

The validation cohort originated from the Utrecht General 
Practitioners Network database, in which collaborating GPs 
enter all data related to daily patient contacts electronically 
into a GP information system. For this study we included 
793 patients aged 50 years or over with a diagnosis of COPD 
based on available spirometric data (post-bronchodilator 
forced expiratory volume in 1 second [FEVJ/forced vital 
capacity <70%). This database can be considered as a 



dynamic cohort. We artificially set the study entry at January 
20 1 0 for our study, and baseline characteristics and follow-up 
data of the eligible patients were determined for 24 months 
from that period of time onwards. 

Outcome 

For both cohorts we used the same "operational" definition 
for exacerbation of COPD, that is, symptomatic deterioration 
requiring pulsed oral steroid use or hospitalization. 15-17 

Candidate predictors 

The list of candidate predictors to be studied was based on 
the literature and was limited to those predictors that are 
readily available in clinical practice. We added predictors 
related to cardiovascular comorbidity. The list of selected 
potential predictors included age, 10 FEV,, 9101819 previous 
exacerbations, 9,1019,20 body mass index, 18-20 pack years 
smoked, history of vascular disease, 21-23 and history of 
ischemic heart disease (IHD). All candidate predictors were 
assessed at study entry for each patient. 

Previous exacerbations were considered present if at least 
one exacerbation occurred in the year prior to the baseline 
assessment, also defined as symptomatic deterioration requir- 
ing pulsed oral steroid use or a hospitalization. For FEV the 
post-bronchodilator value was used and the percentage of 
predicted (%predicted) FEVj according to age, height, and 
sex was calculated. A history of IHD was defined as a his- 
tory of myocardial infarction, angina pectoris, percutaneous 
coronary intervention, or coronary artery bypass grafting. A 
history of vascular disease was defined as stroke, transient 
ischemic attack, or peripheral arterial disease. 

Missing data 

In the validation cohort, follow-up information was missing 
for three participants ( 1 .3%), and these patients were excluded 
from the analyses. In six participants (2.5%), data on previous 
exacerbations were missing. In the validation cohort, post- 
bronchodilator FEVj values were missing in eleven patients 
(1.4%), smoking status in 161 patients (20.3%), andnumber 
of pack years in 398 patients (50.2%). All missing values 
were imputed using multiple regression techniques. 24 Rubin's 
rules 25 were applied to the 25 multiple imputation sets to 
calculate the pooled C-statistics. As the exclusion of patients 
with a missing value (so-called complete-case analysis) in 
general may lead to biased results, we performed a separate 
reanalysis that was restricted to "complete cases" of the 
univariable associations between the number of pack years 
smoked and exacerbations. The univariable odds ratios were 
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similar to those derived from the imputed dataset, although 
with broader confidence intervals (CIs). 

Data analyses 

Model development 

Descriptive analyses were used to assess differences in 
baseline characteristics of patients who experienced an exac- 
erbation in the 2 years after the baseline assessment and those 
who did not. For model development, we used multivariable 
logistic regression analysis to examine the independent 
association between candidate predictors and the occurrence 
of a COPD exacerbation within 24 months. The strength of 
the relationship was expressed in odds ratios with 95% CIs. 
Age, FEVj , and pack years were assessed on their continuous 
scale. To improve linearity, we log-transformed the number 
of pack years smoked. 

We started with all six preselected candidate predictors in 
the model and then applied backward-selection procedures, 
using a i 3 -value<0.20 from the log likelihood ratio test to 
select predictors for the final model. The model was inter- 
nally validated using bootstrapping techniques. To correct for 
overfitting, the beta coefficients of the predictors in the final 
model were multiplied by the shrinkage factor derived from the 
500 bootstrap samples. 26 The performance of the final model 
(after shrinkage) was assessed with the C-statistic and its 95% 
CI, and calibration was assessed with a calibration plot. 

External validation 

For validation of the model, the predicted probabilities were 
calculated for each patient in the validation set using the 



regression coefficients, after shrinkage, from the original 
model obtained from the derivation cohort. In a calibration 
plot, the actual and predicted probabilities were compared 
across the range of predicted risks. Discrimination in the 
validation cohort was assessed with the C-statistic. 

Data were analyzed using SPSS 20.0 for Windows (IBM 
Corporation, Armonk, NY, USA) and R (v 2.9.2; The R 
Foundation for Statistical Computing, Vienna, Austria). 

Results 

During the 24-month follow-up period, 70 (29.2%) patients 
experienced at least one exacerbation of COPD in the deri- 
vation cohort, and 222 (28.0%) in the validation cohort. For 
both cohorts, the number of exacerbations increased with 
increasing severity of COPD and ranged from 22% of patients 
with mild COPD (GOLD I) to 45% in patients with severe 
COPD (GOLD III/IV). Disease classification according to 
the GOLD criteria 13 and other baseline characteristics of 
the 240 patients from the derivation cohort and 793 patients 
from the validation cohort, stratified by occurrence of an 
exacerbation, are summarized in Tables 1 and 2. 

Model development 

In the multivariable analysis, decreased values of FEV i; 
previous exacerbations, pack years of smoking, and vas- 
cular disease were independent predictors of an exacerba- 
tion within 24 months. The final reduced prediction model 
after backward elimination is shown in Table 3. The model 
discriminated well with a C-statistic after shrinkage of 
0.75 (95% CI: 0.69-0.82) and with a shrinkage factor of 



Table I Baseline demographic and clinical characteristics of the derivation and validation cohort, with and without exacerbations 
within 24 months from study entry 



Characteristics 


Derivation cohort 




Validation cohort 






Exacerbation 


No exacerbation 


Exacerbation 


No exacerbation 




(n=70) 


(n=l70) 


(n=222) 


(n=57l) 


Mean age, years (±SD) 


73.6 (±5.2) 


73.3 (±5.0) 


67.9 (±10.0) 


67.2 (±9.7) 


Male, n (%) 


SI (72.9) 


1 1 3 (66.5) 


1 19 (53.6) 


301 (52.7) 


Current smokers, n (%) 


24 (34.3) 


34 (20.0) 


92 (41.4) 


202 (35.4) 


Never smokers, n (%) 


5(7.1) 


31 (18.2) 


1 5 (6.8) 


59 (10.3) 


Pack years, median (IQR) a 


32.8 (18.4-54.0) 


23.3 (4.1-51.8) 


37.0 (23. 0^7.0) 


30.0 (20.0-45.0) 


Mean FEV, %predicted (±SD) b 


64.2 (±20.5) 


74.0 (±20.0) 


65.6 (±18.1) 


72.5 (±16.7) 


Inhaled corticosteroids, n (%) 


56 (80.0) 


102 (60.0) 


56 (25.2) 


123 (21.5) 


Inhaled R2-mimetics, n (%) 


56 (80.0) 


1 1 1 (65.3) 


126 (56.8) 


233 (40.8) 


Inhaled anti-cholinergics, n (%) 


45 (64.3) 


88 (51.8) 


122 (55.0) 


271 (47.5) 


Previous exacerbations, n (%) e 


33 (47.1) 


23 (13.5) 


55 (24.8) 


39 (6.8) 


Ischemic heart disease, n (%) d 


14 (20.0) 


38 (22.4) 


36 (16.2) 


101 (17.7) 


Vascular disease, n (%) e 


10 (14.3) 


12(7.1) 


41 (18.5) 


76 (13.3) 


Notes: a n-395: n-l 16 for exacerbation and 


n-279 for no exacerbation; b n 


-782; n-2 1 8 for exacerbation and 


n-564 for no exacerbation; 


c previous exacerbations = 


occurrence of an exacerbation in the year before study entry; ischemic heart d 


sease = history of myocardial infarct! 


on, angina pectoris, percutaneous coronary intervention, 


or coronary artery bypass grafting; e vascular di 


sease = history of stroke, minor 


stroke, or peripheral arterial disease. 




Abbreviations: FEV |t forced expiratory volume in 1 second; IQR, interquartil 


e range; SD, standard deviation. 
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Table 2 Disease classification according to the Global initiative for chronic Obstructive Lung Disease (GOLD) criteria 13 for both 
cohorts, with and without exacerbations within 24 months from baseline assessment 



GOLD classification 



Derivation cohort 



Exacerbation 
(n=70) 



No exacerbation 
(n=!70) 



Validation cohort 3 



Exacerbation 
(n=220) 



No exacerbation 
(n=564) 



GOLD I, n (%) 
GOLD II, n (%) 
GOLD III, n (%) 
GOLD IV, n (%) 



14 (20.0) 
35 (50.0) 
21 (30.0) 
0 (0.0) 



66 (38.8) 
81 (47.6) 
23 (13.5) 
0 (0.0) 



54 (24.5) 
121 (55.0) 
40 (18.2) 
5 (2.3) 



181 (32.1) 
327 (58.0) 
53 (9.4) 
3 (0.5) 



Notes: GOLD I: %predicted forced expiratory volume in I second (FEV,) >80%; GOLD II: 50%< %predicted FEV 5=80%; GOLD 
GOLD IV: %predicted FEV ( ^30%. Classification according to the GOLD criteria missing in nine patients. 



30%< ^predicted FEV, <50%; 



0.90. Calibration of the model was good: the predicted and 
actual risks of exacerbation within 24 months were in close 
agreement (Figure 1). 

Model validation 

The model showed a moderate discrimination in the validation 
cohort, with a C-statistic of 0.66 (95% CI: 0.62-0.71). The 
calibration plot indicated close agreement between the 
predicted and observed risks (Figure 1). 

Discussion 

Our study shows that easy-to-assess variables are inde- 
pendently associated with the occurrence of future COPD 
exacerbations. A model including these variables from 
history-taking and spirometry (FEV, %predicted) could be 
used by both pulmonologists and GPs in everyday care. After 
external validation, our model showed moderate discrimination 
(C-statistic 0.66) and a good calibration. 



Table 3 Final prediction model to predict exacerbations in 
COPD patients 

OR* 95% CI* 

Previous exacerbations 5.07 2.55-10.07 

FEV,, %predicted (OR per 5%) b 0.95 0.88-1 .03 

Pack years of smoking (2log transformed) 1 1.16 1 .0 1 - 1 .35 

History of vascular disease 1 .92 0.89-4. 1 2 

Notes: 'After adjustment for overfitting by shrinkage (shrinkage factor —0.90). b OR 
for each interval change of 5%; 'doubling of the number of log-transformed pack 
years results in 1.16 times higher OR. Risk of COPD exacerbation within the next 
24months -1/(1 +exp-(- 1.33+1 .62xpreviousexacerbation-0.05xFEV (%predicted, 
per 5% interval change) + 0. 1 5 x 2log(pack years) + 0.65 x history vascular disease)). 
Examples: a male person aged 73 years with a history of exacerbations in the 
previous year, FEV, of 32.9 as %predicted, a history of 38 pack years of smoking, 
and prior stroke. His chance of an exacerbation in the next 2 years is: 1/(1 + exp — 
(-1.33 + 1 .62 - 0.05 x (32.9/5) + 0. 1 5 x 2log(38) + 0.65)) = 0.79 = 79%. A female 
person aged 77 years, without any exacerbations in the previous year, FEV! of 77.7 
as %predicted, who never smoked, and who does not have a history of stroke or 
peripheral artery disease. Her chance of an exacerbation in the next 2 years is: 
I /( I + exp - (- 1 .33 + 0 - 0.05 x (77.7/5) + 0. 1 5 x 2log(0) + 0)) = 0.09 = 9%. 
Abbreviations: CI, confidence interval; COPD, chronic obstructive pulmonary 
disease: FEV ( , forced expiratory volume in I second: OR, odds ratio. 



Our findings that lower levels of FEV, as %predicted 
and that a history of previous exacerbations are related to an 
increased risk of future exacerbations are in line with previous 
studies. 9,10,18 ~ 20 Hurst et al 20 identified previous exacerbations 
as the single best predictor of future exacerbations. To exam- 
ine this claim in our cohort, we tested previous exacerbations 
as a single value, which resulted in C-statistics of 0.66 and 
0.59 in the derivation and validation cohort, respectively. This 
suggests that other predictors are needed to more adequately 
predict future exacerbations in a broad sample of patients 
with COPD. We also evaluated the DOSE index in our deriva- 
tion cohort. 11 This index yielded a C-statistic of 0.65, which 
is considerably lower than our own model. Unfortunately, we 
could not evaluate the widely accepted BODE index, 5 because 
we did not perform a 6-minute walking test in any of our 
cohorts. Importantly, however, when we applied the BODE 
index without information on exercise capacity, it resulted in 
a poor C-statistic of 0.6 1 in the derivation cohort. Moreover, 
the performance of the BODE index on prediction of future 
exacerbations is rather poor in other than the original cohorts, 
with C-statistics of 0.67 and 0.62. 6 

We also considered cardiovascular diseases because they 
are a major cause of morbidity and mortality in patients with 
COPD. 27-30 Low-grade systemic inflammation and smok- 
ing are key components in the pathophysiology of vascular 
plaque formation, which may explain the increased rates of 
cardiovascular disease in COPD. 31 Importantly, however, 
COPD patients with concomitant IHD did not experience a 
higher incidence of COPD exacerbations in our population, 
a result similar to previous studies. 30,32 On the other hand, 
a history of vascular disease (including stroke) was positively 
(and independently) associated with COPD exacerbations in 
our study. Although the relationship between deep venous 
thrombosis and exacerbations is well studied, 21-23 vascu- 
lar disease at large, including peripheral arterial disease 
and stroke, has not been identified before as a predictor 
of future COPD exacerbations. Peripheral arterial disease 
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Figure I Calibration plots of the final prediction model for exacerbations of COPD within the proceeding 24 months in both the derivation and validation cohort. 
Notes: Left: calibration plot in the derivation cohort. Right: calibration plot in the validation cohort. The dotted line shows the actual relation between observed outcomes 
and predicted risks; the solid line shows the smoothed relation. Ideally, these lines equal the dashed diagonal line that represents perfect calibration. 
Abbreviation: COPD, chronic obstructive pulmonary disease. 



is possibly associated with more severe COPD, 33 and this 
might translate into the observed association between vas- 
cular disease and the risk of future exacerbations. Increased 
platelet activation could be a mechanism that plays a role in 
this association. 34 

We were not able to include all potential predictors 
available from the literature, such as exercise capacity, the 
St George's Respiratory Questionnaire, oxygen therapy, 
and gastroesophageal reflux, because these variables were 
not available in the derivation cohort. Importantly, however, 
most of these variables are not available in clinical practice, 
and, thereby, these predictors would not be useful for predic- 
tion within individual patients in everyday practice. 

It has become increasingly clear that the severity and fre- 
quency of exacerbations is only marginally related to disease 
severity. Because most patients with COPD can be classified 
as GOLD I and II (ie, mild COPD), the absolute number of 
exacerbations and overall burden is largest in those with mild 
COPD. 20 This illustrates the importance of including patients 
with GOLD I and II, as we did in our study. 

Patients in the validation cohort were younger; more 
frequently current smokers; had, in general, milder disease; 
were more likely to have vascular disease; and experienced 
fewer exacerbations in the previous year than the derivation 
cohort. Despite these differences in baseline characteristics 
("case mix"), the model yielded acceptable discrimina- 
tion (C-statistic 0.66) and good calibration, indicating 
that our findings are generally applicable, including in 
younger COPD patients. The relatively small sample size 



of the derivation cohort could be considered a limitation 
of the study. We chose this data set because the data were 
prospectively collected, with, as a result, a higher quality of 
data compared to the validation cohort. 

The definition of COPD exacerbations is always a chal- 
lenge in follow-up studies such as ours. We used symptomatic 
deterioration requiring pulsed oral steroid use or hospital- 
ization as a definition for exacerbation. This is somewhat 
different from previous studies that also included treatment 
with antibiotics. 35 " 37 In Dutch primary care guidelines on 
COPD, short-course oral corticosteroids are recommended 
for COPD exacerbations, and addition of antibiotics should 
only be considered when bacterial infection is suspected. 38 
Thus, exacerbations of COPD are rarely, if ever, treated 
solely with antibiotics only in the Netherlands. As a result, 
with our definition, all exacerbations are captured in the 
derivation and validation cohort and, as such, our results are 
generalizable to other populations of COPD, including in 
other parts of the world. 

We used logistic regression modeling to estimate the risk 
of occurrence of COPD exacerbations within the proceeding 
24 months. Cox proportional hazards modeling would also 
have been a valid approach for risk estimation, but we chose 
logistic regression analysis because we considered each exac- 
erbation within our predefined time frame of 2 years to be 
of equal importance, regardless of whether this exacerbation 
occurred early or late in the follow-up period. Furthermore, 
calculation of absolute probabilities of exacerbation in future 
patients is more straightforward using the coefficients of a 
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logistic model than it would be for a Cox survival model due 
to the unspecified baseline function in the latter. 39 

To the best of our knowledge, this is the first study provid- 
ing predictors of future exacerbations in a broad spectrum of 
patients with COPD, including patients with mild disease. 
Because we developed and tested the model in a sample of 
the COPD patient population at large, the applicability of 
the model is broad and seems applicable to both primary and 
secondary care. The performance of our prediction model is 
very promising, and the applicability of the model is facili- 
tated by the limited number of predictors in the model and 
their ready availability in daily practice. The model helps 
clinicians to identify COPD patients at high risk of exacerba- 
tions and to guide intensified management and monitoring 
of these patients. To determine the impact of our prediction 
model on daily practice in terms of patient outcome and the 
use of health care resources, an implementation study should 
be performed. 

Conclusion 

A limited number of easily assessable variables adequately 
predict future exacerbations of COPD. Cardiovascular 
disease should be considered in the management of these 
patients, and adequate treatment of vascular disease could 
possibly reduce the risk of future exacerbations. 
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